Ambient cooperative intelligence system and method

ABSTRACT

A method, computer program product, and computing system for generating a three-dimensional model of at least a portion of a three-dimensional space incorporating an ACI system via a video recording subsystem of an ACI calibration platform; and generating one or more audio calibration signals for receipt by an audio recording system included within the ACI system via an audio generation subsystem of the ACI calibration platform.

RELATED APPLICATION(S)

This application is a continuation of Non-Provisional application Ser.No. 17/077,863, filed 22 Oct. 2020, which claims the benefit of U.S.Provisional Application No. 62/988,337, filed on 11 Mar. 2020, theentire contents of which are herein incorporated by reference.

TECHNICAL FIELD

This disclosure relates to intelligence systems and methods and, moreparticularly, to ambient cooperative intelligence systems and methods.

BACKGROUND

As is known in the art, cooperative intelligence is the creation ofreports and documentation that details the history of anevent/individual. As would be expected, traditional documentationincludes various types of data, examples of which may include but arenot limited to paper-based documents and transcripts, as well as variousimages and diagrams.

As the world moved from paper-based content to digital content,traditional documentation also moved in that direction, where reportsand documentation were gradually transitioned from stacks of papergeographically-dispersed across multiple locations/institutions toconsolidated and readily accessible digital content.

SUMMARY OF DISCLOSURE

In one implementation, a computer-implemented method is executed on acomputing device and includes: generating a three-dimensional model ofat least a portion of a three-dimensional space incorporating an ACIsystem via a video recording subsystem of an ACI calibration platform;and generating one or more audio calibration signals for receipt by anaudio recording system included within the ACI system via an audiogeneration subsystem of the ACI calibration platform.

One or more of the following features may be included. The ACIcalibration platform may be autonomously positioned within thethree-dimensional space via a mobile base assembly of the ACIcalibration platform. At least a portion of the three-dimensional spacemay be autonomously cleaned via a cleaning assembly of the ACIcalibration platform. The ACI calibration platform may be configured tobe manually positioned within the three-dimensional space. The videorecording system of the ACI calibration platform may be configured tointerface with an object datasource that defines a plurality of objectsthat may be located within the three-dimensional space. Thethree-dimensional model may be configured to define at least one of: oneor more subspaces within the three-dimensional space; one or moreobjects within the three-dimensional space; one or more features withinthe three-dimensional space; one or more interaction zones within thethree-dimensional space; and one or more noise sources within thethree-dimensional space. The one or more audio calibration signals mayinclude one or more of: a noise signal; a sinusoid signal; and amulti-frequency signal.

In another implementation, a computer program product resides on acomputer readable medium and has a plurality of instructions stored onit. When executed by a processor, the instructions cause the processorto perform operations including: generating a three-dimensional model ofat least a portion of a three-dimensional space incorporating an ACIsystem via a video recording subsystem of an ACI calibration platform;and generating one or more audio calibration signals for receipt by anaudio recording system included within the ACI system via an audiogeneration subsystem of the ACI calibration platform.

One or more of the following features may be included. The ACIcalibration platform may be autonomously positioned within thethree-dimensional space via a mobile base assembly of the ACIcalibration platform. At least a portion of the three-dimensional spacemay be autonomously cleaned via a cleaning assembly of the ACIcalibration platform. The ACI calibration platform may be configured tobe manually positioned within the three-dimensional space. The videorecording system of the ACI calibration platform may be configured tointerface with an object datasource that defines a plurality of objectsthat may be located within the three-dimensional space. Thethree-dimensional model may be configured to define at least one of: oneor more subspaces within the three-dimensional space; one or moreobjects within the three-dimensional space; one or more features withinthe three-dimensional space; one or more interaction zones within thethree-dimensional space; and one or more noise sources within thethree-dimensional space. The one or more audio calibration signals mayinclude one or more of: a noise signal; a sinusoid signal; and amulti-frequency signal.

In another implementation, a computing system includes a processor andmemory is configured to perform operations including: generating athree-dimensional model of at least a portion of a three-dimensionalspace incorporating an ACI system via a video recording subsystem of anACI calibration platform; and generating one or more audio calibrationsignals for receipt by an audio recording system included within the ACIsystem via an audio generation subsystem of the ACI calibrationplatform.

One or more of the following features may be included. The ACIcalibration platform may be autonomously positioned within thethree-dimensional space via a mobile base assembly of the ACIcalibration platform. At least a portion of the three-dimensional spacemay be autonomously cleaned via a cleaning assembly of the ACIcalibration platform. The ACI calibration platform may be configured tobe manually positioned within the three-dimensional space. The videorecording system of the ACI calibration platform may be configured tointerface with an object datasource that defines a plurality of objectsthat may be located within the three-dimensional space. Thethree-dimensional model may be configured to define at least one of: oneor more subspaces within the three-dimensional space; one or moreobjects within the three-dimensional space; one or more features withinthe three-dimensional space; one or more interaction zones within thethree-dimensional space; and one or more noise sources within thethree-dimensional space. The one or more audio calibration signals mayinclude one or more of: a noise signal; a sinusoid signal; and amulti-frequency signal.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features andadvantages will become apparent from the description, the drawings, andthe claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic view of an ambient cooperative intelligencecompute system and an ambient cooperative intelligence process coupledto a distributed computing network;

FIG. 2 is a diagrammatic view of a modular ACI system incorporating theambient cooperative intelligence compute system of FIG. 1;

FIG. 3 is a diagrammatic view of a mixed-media ACI device includedwithin the modular ACI system of FIG. 2;

FIG. 4 is a flow chart of one implementation of the ambient cooperativeintelligence process of FIG. 1;

FIG. 5 is a flow chart of another implementation of the ambientcooperative intelligence process of FIG. 1;

FIG. 6 is a flow chart of another implementation of the ambientcooperative intelligence process of FIG. 1;

FIG. 7 is a flow chart of another implementation of the ambientcooperative intelligence process of FIG. 1;

FIG. 8 is a diagrammatic view of an ACI calibration platform;

FIG. 9 is a flow chart of one implementation of a process executed bythe ACI calibration platform of FIG. 8; and

FIG. 10 is a flow chart of another implementation of the ambientcooperative intelligence process of FIG. 1.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS System Overview

Referring to FIG. 1, there is shown ambient cooperative intelligenceprocess 10. As will be discussed below in greater detail, ambientcooperative intelligence process 10 may be configured to automate thecollection and processing of encounter information togenerate/store/distribute reports.

Ambient cooperative intelligence process 10 may be implemented as aserver-side process, a client-side process, or a hybridserver-side/client-side process. For example, ambient cooperativeintelligence process 10 may be implemented as a purely server-sideprocess via ambient cooperative intelligence process 10 s.Alternatively, ambient cooperative intelligence process 10 may beimplemented as a purely client-side process via one or more of ambientcooperative intelligence process 10 c 1, ambient cooperativeintelligence process 10 c 2, ambient cooperative intelligence process 10c 3, and ambient cooperative intelligence process 10 c 4. Alternativelystill, ambient cooperative intelligence process 10 may be implemented asa hybrid server-side/client-side process via ambient cooperativeintelligence process 10 s in combination with one or more of ambientcooperative intelligence process 10 c 1, ambient cooperativeintelligence process 10 c 2, ambient cooperative intelligence process 10c 3, and ambient cooperative intelligence process 10 c 4.

Accordingly, ambient cooperative intelligence process 10 as used in thisdisclosure may include any combination of ambient cooperativeintelligence process 10 s, ambient cooperative intelligence process 10 c1, ambient cooperative intelligence process 10 c 2, ambient cooperativeintelligence process 10 c 3, and ambient cooperative intelligenceprocess 10 c 4.

Ambient cooperative intelligence process 10 s may be a serverapplication and may reside on and may be executed by ambient cooperativeintelligence (ACI) compute system 12, which may be connected to network14 (e.g., the Internet or a local area network). ACI compute system 12may include various components, examples of which may include but arenot limited to: a personal computer, a server computer, a series ofserver computers, a mini computer, a mainframe computer, one or moreNetwork Attached Storage (NAS) systems, one or more Storage Area Network(SAN) systems, one or more Platform as a Service (PaaS) systems, one ormore Infrastructure as a Service (IaaS) systems, one or more Software asa Service (SaaS) systems, a cloud-based computational system, and acloud-based storage platform.

As is known in the art, a SAN may include one or more of a personalcomputer, a server computer, a series of server computers, a minicomputer, a mainframe computer, a RAID device and a NAS system. Thevarious components of ACI compute system 12 may execute one or moreoperating systems, examples of which may include but are not limited to:Microsoft Windows Server™; Redhat Linux™, Unix, or a custom operatingsystem, for example.

The instruction sets and subroutines of ambient cooperative intelligenceprocess 10 s, which may be stored on storage device 16 coupled to ACIcompute system 12, may be executed by one or more processors (not shown)and one or more memory architectures (not shown) included within ACIcompute system 12. Examples of storage device 16 may include but are notlimited to: a hard disk drive; a RAID device; a random access memory(RAM); a read-only memory (ROM); and all forms of flash memory storagedevices.

Network 14 may be connected to one or more secondary networks (e.g.,network 18), examples of which may include but are not limited to: alocal area network; a wide area network; or an intranet, for example.

Various IO requests (e.g. IO request 20) may be sent from ambientcooperative intelligence process 10 s, ambient cooperative intelligenceprocess 10 c 1, ambient cooperative intelligence process 10 c 2, ambientcooperative intelligence process 10 c 3 and/or ambient cooperativeintelligence process 10 c 4 to ACI compute system 12. Examples of IOrequest 20 may include but are not limited to data write requests (i.e.a request that content be written to ACI compute system 12) and dataread requests (i.e. a request that content be read from ACI computesystem 12).

The instruction sets and subroutines of ambient cooperative intelligenceprocess 10 c 1, ambient cooperative intelligence process 10 c 2, ambientcooperative intelligence process 10 c 3 and/or ambient cooperativeintelligence process 10 c 4, which may be stored on storage devices 20,22, 24, 26 (respectively) coupled to ACI client electronic devices 28,30, 32, 34 (respectively), may be executed by one or more processors(not shown) and one or more memory architectures (not shown)incorporated into ACI client electronic devices 28, 30, 32, 34(respectively). Storage devices 20, 22, 24, 26 may include but are notlimited to: hard disk drives; optical drives; RAID devices; randomaccess memories (RAM); read-only memories (ROM), and all forms of flashmemory storage devices. Examples of ACI client electronic devices 28,30, 32, 34 may include, but are not limited to, personal computingdevice 28 (e.g., a smart phone, a personal digital assistant, a laptopcomputer, a notebook computer, and a desktop computer), audio inputdevice 30 (e.g., a handheld microphone, a lapel microphone, an embeddedmicrophone (such as those embedded within eyeglasses, smart phones,tablet computers and/or watches) and an audio recording device), displaydevice 32 (e.g., a tablet computer, a computer monitor, and a smarttelevision), machine vision input device 34 (e.g., an RGB imagingsystem, an infrared imaging system, an ultraviolet imaging system, alaser imaging system, a SONAR imaging system, a RADAR imaging system,and a thermal imaging system), a hybrid device (e.g., a single devicethat includes the functionality of one or more of the above-referencesdevices; not shown), an audio rendering device (e.g., a speaker system,a headphone system, or an earbud system; not shown), various medicaldevices (e.g., medical imaging equipment, heart monitoring machines,body weight scales, body temperature thermometers, and blood pressuremachines; not shown), and a dedicated network device (not shown).

Users 36, 38, 40, 42 may access ACI compute system 12 directly throughnetwork 14 or through secondary network 18. Further, ACI compute system12 may be connected to network 14 through secondary network 18, asillustrated with link line 44.

The various ACI client electronic devices (e.g., ACI client electronicdevices 28, 30, 32, 34) may be directly or indirectly coupled to network14 (or network 18). For example, personal computing device 28 is showndirectly coupled to network 14 via a hardwired network connection.Further, machine vision input device 34 is shown directly coupled tonetwork 18 via a hardwired network connection. Audio input device 30 isshown wirelessly coupled to network 14 via wireless communicationchannel 46 established between audio input device 30 and wireless accesspoint (i.e., WAP) 48, which is shown directly coupled to network 14. WAP48 may be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802.11n,Wi-Fi, and/or Bluetooth device that is capable of establishing wirelesscommunication channel 46 between audio input device 30 and WAP 48.Display device 32 is shown wirelessly coupled to network 14 via wirelesscommunication channel 50 established between display device 32 and WAP52, which is shown directly coupled to network 14.

The various ACI client electronic devices (e.g., ACI client electronicdevices 28, 30, 32, 34) may each execute an operating system, examplesof which may include but are not limited to Microsoft Windows™, AppleMacintosh™, Redhat Linux™, or a custom operating system, wherein thecombination of the various ACI client electronic devices (e.g., ACIclient electronic devices 28, 30, 32, 34) and ACI compute system 12 mayform modular ACI system 54.

The Ambient Cooperative Intelligence System

While ambient cooperative intelligence process 10 will be describedbelow as being utilized to automate the collection and processing ofclinical encounter information to generate/store/distribute medicalrecords, this is for illustrative purposes only and is not intended tobe a limitation of this disclosure, as other configurations are possibleand are considered to be within the scope of this disclosure.

Referring also to FIG. 2, there is shown a simplified exemplaryembodiment of modular ACI system 54 that is configured to automatecooperative intelligence. Modular ACI system 54 may include: machinevision system 100 configured to obtain machine vision encounterinformation 102 concerning a patient encounter; audio recording system104 configured to obtain audio encounter information 106 concerning thepatient encounter; and a compute system (e.g., ACI compute system 12)configured to receive machine vision encounter information 102 and audioencounter information 106 from machine vision system 100 and audiorecording system 104 (respectively). Modular ACI system 54 may alsoinclude: display rendering system 108 configured to render visualinformation 110; and audio rendering system 112 configured to renderaudio information 114, wherein ACI compute system 12 may be configuredto provide visual information 110 and audio information 114 to displayrendering system 108 and audio rendering system 112 (respectively).

Example of machine vision system 100 may include but are not limited to:one or more ACI client electronic devices (e.g., ACI client electronicdevice 34, examples of which may include but are not limited to an RGBimaging system, an infrared imaging system, a ultraviolet imagingsystem, a laser imaging system, a SONAR imaging system, a RADAR imagingsystem, and a thermal imaging system). Examples of audio recordingsystem 104 may include but are not limited to: one or more ACI clientelectronic devices (e.g., ACI client electronic device 30, examples ofwhich may include but are not limited to a handheld microphone (e.g.,one example of a body worn microphone), a lapel microphone (e.g.,another example of a body worn microphone), an embedded microphone, suchas those embedded within eyeglasses, smart phones, tablet computersand/or watches (e.g., another example of a body worn microphone), and anaudio recording device). Examples of display rendering system 108 mayinclude but are not limited to: one or more ACI client electronicdevices (e.g., ACI client electronic device 32, examples of which mayinclude but are not limited to a tablet computer, a computer monitor,and a smart television). Examples of audio rendering system 112 mayinclude but are not limited to: one or more ACI client electronicdevices (e.g., audio rendering device 116, examples of which may includebut are not limited to a speaker system, a headphone system, and anearbud system).

ACI compute system 12 may be configured to access one or moredatasources 118 (e.g., plurality of individual datasources 120, 122,124, 126, 128), examples of which may include but are not limited to oneor more of a user profile datasource, a voice print datasource, a voicecharacteristics datasource (e.g., for adapting the ambient speechrecognition models), a face print datasource, a humanoid shapedatasource, an utterance identifier datasource, a wearable tokenidentifier datasource, an interaction identifier datasource, a medicalconditions symptoms datasource, a prescriptions compatibilitydatasource, a medical insurance coverage datasource, a physical eventsdatasource, and a home healthcare datasource. While in this particularexample, five different examples of datasources 118 are shown, this isfor illustrative purposes only and is not intended to be a limitation ofthis disclosure, as other configurations are possible and are consideredto be within the scope of this disclosure.

As will be discussed below in greater detail, modular ACI system 54 maybe configured to monitor a monitored space (e.g., monitored space 130)in a clinical environment, wherein examples of this clinical environmentmay include but are not limited to: a doctor's office, a medicalfacility, a medical practice, a medical lab, an urgent care facility, amedical clinic, an emergency room, an operating room, a hospital, a longterm care facility, a rehabilitation facility, a nursing home, and ahospice facility. Accordingly, an example of the above-referencedpatient encounter may include but is not limited to a patient visitingone or more of the above-described clinical environments (e.g., adoctor's office, a medical facility, a medical practice, a medical lab,an urgent care facility, a medical clinic, an emergency room, anoperating room, a hospital, a long term care facility, a rehabilitationfacility, a nursing home, and a hospice facility).

Machine vision system 100 may include a plurality of discrete machinevision systems when the above-described clinical environment is largeror a higher level of resolution is desired. As discussed above, examplesof machine vision system 100 may include but are not limited to: one ormore ACI client electronic devices (e.g., ACI client electronic device34, examples of which may include but are not limited to an RGB imagingsystem, an infrared imaging system, an ultraviolet imaging system, alaser imaging system, a SONAR imaging system, a RADAR imaging system,and a thermal imaging system). Accordingly, machine vision system 100may include one or more of each of an RGB imaging system, an infraredimaging systems, an ultraviolet imaging systems, a laser imaging system,a SONAR imaging system, a RADAR imaging system, and a thermal imagingsystem.

Audio recording system 104 may include a plurality of discrete audiorecording systems when the above-described clinical environment islarger or a higher level of resolution is desired. As discussed above,examples of audio recording system 104 may include but are not limitedto: one or more ACI client electronic devices (e.g., ACI clientelectronic device 30, examples of which may include but are not limitedto a handheld microphone, a lapel microphone, an embedded microphone(such as those embedded within eyeglasses, smart phones, tabletcomputers and/or watches) and an audio recording device). Accordingly,audio recording system 104 may include one or more of each of a handheldmicrophone, a lapel microphone, an embedded microphone (such as thoseembedded within eyeglasses, smart phones, tablet computers and/orwatches) and an audio recording device.

Display rendering system 108 may include a plurality of discrete displayrendering systems when the above-described clinical environment islarger or a higher level of resolution is desired. As discussed above,examples of display rendering system 108 may include but are not limitedto: one or more ACI client electronic devices (e.g., ACI clientelectronic device 32, examples of which may include but are not limitedto a tablet computer, a computer monitor, and a smart television).Accordingly, display rendering system 108 may include one or more ofeach of a tablet computer, a computer monitor, and a smart television.

Audio rendering system 112 may include a plurality of discrete audiorendering systems when the above-described clinical environment islarger or a higher level of resolution is desired. As discussed above,examples of audio rendering system 112 may include but are not limitedto: one or more ACI client electronic devices (e.g., audio renderingdevice 116, examples of which may include but are not limited to aspeaker system, a headphone system, or an earbud system). Accordingly,audio rendering system 112 may include one or more of each of a speakersystem, a headphone system, or an earbud system.

ACI compute system 12 may include a plurality of discrete computesystems. As discussed above, ACI compute system 12 may include variouscomponents, examples of which may include but are not limited to: apersonal computer, a server computer, a series of server computers, amini computer, a mainframe computer, one or more Network AttachedStorage (NAS) systems, one or more Storage Area Network (SAN) systems,one or more Platform as a Service (PaaS) systems, one or moreInfrastructure as a Service (IaaS) systems, one or more Software as aService (SaaS) systems, a cloud-based computational system, and acloud-based storage platform. Accordingly, ACI compute system 12 mayinclude one or more of each of a personal computer, a server computer, aseries of server computers, a mini computer, a mainframe computer, oneor more Network Attached Storage (NAS) systems, one or more Storage AreaNetwork (SAN) systems, one or more Platform as a Service (PaaS) systems,one or more Infrastructure as a Service (IaaS) systems, one or moreSoftware as a Service (SaaS) systems, a cloud-based computationalsystem, and a cloud-based storage platform.

Microphone Array

Referring also to FIG. 3, audio recording system 104 may includemicrophone array 200 having a plurality of discrete microphoneassemblies. For example, audio recording system 104 may include aplurality of discrete audio acquisition devices (e.g., audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218) that may formmicrophone array 200. As will be discussed below in greater detail,modular ACI system 54 may be configured to form one or more audiorecording beams (e.g., audio recording beams 220, 222, 224) via thediscrete audio acquisition devices (e.g., audio acquisition devices 202,204, 206, 208, 210, 212, 214, 216, 218) included within audio recordingsystem 104.

For example, modular ACI system 54 may be further configured to steerthe one or more audio recording beams (e.g., audio recording beams 220,222, 224) toward one or more encounter participants (e.g., encounterparticipants 226, 228, 230) of the above-described patient encounter.Examples of the encounter participants (e.g., encounter participants226, 228, 230) may include but are not limited to: medical professionals(e.g., doctors, nurses, physician's assistants, lab technicians,physical therapists, scribes (e.g., a transcriptionist) and/or staffmembers involved in the patient encounter), patients (e.g., people thatare visiting the above-described clinical environments for the patientencounter), and third parties (e.g., friends of the patient, relativesof the patient and/or acquaintances of the patient that are involved inthe patient encounter).

Accordingly, modular ACI system 54 and/or audio recording system 104 maybe configured to utilize one or more of the discrete audio acquisitiondevices (e.g., audio acquisition devices 202, 204, 206, 208, 210, 212,214, 216, 218) to form an audio recording beam. For example, modular ACIsystem 54 and/or audio recording system 104 may be configured to utilizevarious audio acquisition devices to form audio recording beam 220, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 226 (as audio recording beam 220 is pointed to (i.e.,directed toward) encounter participant 226). Additionally, modular ACIsystem 54 and/or audio recording system 104 may be configured to utilizevarious audio acquisition devices to form audio recording beam 222, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 228 (as audio recording beam 222 is pointed to (i.e.,directed toward) encounter participant 228). Additionally, modular ACIsystem 54 and/or audio recording system 104 may be configured to utilizevarious audio acquisition devices to form audio recording beam 224, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 230 (as audio recording beam 224 is pointed to (i.e.,directed toward) encounter participant 230).

Further, modular ACI system 54 and/or audio recording system 104 may beconfigured to utilize null-steering precoding to cancel interferencebetween speakers and/or noise. As is known in the art, null-steeringprecoding is a method of spatial signal processing by which a multipleantenna transmitter may null multiuser interference signals in wirelesscommunications, wherein null-steering precoding may mitigate the impactoff background noise and unknown user interference. In particular,null-steering precoding may be a method of beamforming for narrowbandsignals that may compensate for delays of receiving signals from aspecific source at different elements of an antenna array. In generaland to improve performance of the antenna array, incoming signals may besummed and averaged, wherein certain signals may be weighted andcompensation may be made for signal delays.

Machine vision system 100 and audio recording system 104 may bestand-alone devices (as shown in FIG. 2). Additionally/alternatively,machine vision system 100 and audio recording system 104 may be combinedinto one package to form mixed-media ACI device 232. For example,mixed-media ACI device 232 may be configured to be mounted to astructure (e.g., a wall, a ceiling, a beam, a column) within theabove-described clinical environments (e.g., a doctor's office, amedical facility, a medical practice, a medical lab, an urgent carefacility, a medical clinic, an emergency room, an operating room, ahospital, a long term care facility, a rehabilitation facility, anursing home, and a hospice facility), thus allowing for easyinstallation of the same. Further, modular ACI system 54 may beconfigured to include a plurality of mixed-media ACI devices (e.g.,mixed-media ACI device 232) when the above-described clinicalenvironment is larger or a higher level of resolution is desired.

Modular ACI system 54 may be further configured to steer the one or moreaudio recording beams (e.g., audio recording beams 220, 222, 224) towardone or more encounter participants (e.g., encounter participants 226,228, 230) of the patient encounter based, at least in part, upon machinevision encounter information 102. As discussed above, mixed-media ACIdevice 232 (and machine vision system 100/audio recording system 104included therein) may be configured to monitor one or more encounterparticipants (e.g., encounter participants 226, 228, 230) of a patientencounter.

Specifically and as will be discussed below in greater detail, machinevision system 100 (either as a stand-alone system or as a component ofmixed-media ACI device 232) may be configured to detect humanoid shapeswithin the above-described clinical environments (e.g., a doctor'soffice, a medical facility, a medical practice, a medical lab, an urgentcare facility, a medical clinic, an emergency room, an operating room, ahospital, a long term care facility, a rehabilitation facility, anursing home, and a hospice facility). And when these humanoid shapesare detected by machine vision system 100, modular ACI system 54 and/oraudio recording system 104 may be configured to utilize one or more ofthe discrete audio acquisition devices (e.g., audio acquisition devices202, 204, 206, 208, 210, 212, 214, 216, 218) to form an audio recordingbeam (e.g., audio recording beams 220, 222, 224) that is directed towardeach of the detected humanoid shapes (e.g., encounter participants 226,228, 230).

As discussed above, ACI compute system 12 may be configured to receivemachine vision encounter information 102 and audio encounter information106 from machine vision system 100 and audio recording system 104(respectively); and may be configured to provide visual information 110and audio information 114 to display rendering system 108 and audiorendering system 112 (respectively). Depending upon the manner in whichmodular ACI system 54 (and/or mixed-media ACI device 232) is configured,ACI compute system 12 may be included within mixed-media ACI device 232or external to mixed-media ACI device 232.

The Ambient Cooperative Intelligence Process

As discussed above, ACI compute system 12 may execute all or a portionof ambient cooperative intelligence process 10, wherein the instructionsets and subroutines of ambient cooperative intelligence process 10(which may be stored on one or more of e.g., storage devices 16, 20, 22,24, 26) may be executed by ACI compute system 12 and/or one or more ofACI client electronic devices 28, 30, 32, 34.

As discussed above, ambient cooperative intelligence process 10 may beconfigured to automate the collection and processing of clinicalencounter information to generate/store/distribute medical records.Accordingly and referring also to FIG. 4, ambient cooperativeintelligence process 10 may be configured to obtain 300 encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) of a patient encounter (e.g., a visit to adoctor's office). Ambient cooperative intelligence process 10 mayfurther be configured to process 302 the encounter information (e.g.,machine vision encounter information 102 and/or audio encounterinformation 106) to generate an encounter transcript (e.g., encountertranscript 234), wherein ambient cooperative intelligence process 10 maythen process 304 at least a portion of the encounter transcript (e.g.,encounter transcript 234) to populate at least a portion of a medicalrecord (e.g., medical record 236) associated with the patient encounter(e.g., the visit to the doctor's office). Encounter transcript 234and/or medical record 236 may be reviewed by a medical professionalinvolved with the patient encounter (e.g., a visit to a doctor's office)to determine the accuracy of the same and/or make corrections to thesame.

For example, a scribe involved with (or assigned to) the patientencounter (e.g., a visit to a doctor's office) may review encountertranscript 234 and/or medical record 236 to confirm that the same wasaccurate and/or make corrections to the same. In the event thatcorrections are made to encounter transcript 234 and/or medical record236, ambient cooperative intelligence process 10 may utilize thesecorrections for training/tuning purposes (e.g., to adjust the variousprofiles associated the participants of the patient encounter) toenhance the future accuracy/efficiency/performance of ambientcooperative intelligence process 10.

Alternatively/additionally, a doctor involved with the patient encounter(e.g., a visit to a doctor's office) may review encounter transcript 234and/or medical record 236 to confirm that the same was accurate and/ormake corrections to the same. In the event that corrections are made toencounter transcript 234 and/or medical record 236, ambient cooperativeintelligence process 10 may utilize these corrections fortraining/tuning purposes (e.g., to adjust the various profilesassociated the participants of the patient encounter) to enhance thefuture accuracy/efficiency/performance of ambient cooperativeintelligence process 10.

For example, assume that a patient (e.g., encounter participant 228)visits a clinical environment (e.g., a doctor's office) because they donot feel well. They have a headache, fever, chills, a cough, and somedifficulty breathing. In this particular example, a monitored space(e.g., monitored space 130) within the clinical environment (e.g., thedoctor's office) may be outfitted with machine vision system 100configured to obtain machine vision encounter information 102 concerningthe patient encounter (e.g., encounter participant 228 visiting thedoctor's office) and audio recording system 104 configured to obtainaudio encounter information 106 concerning the patient encounter (e.g.,encounter participant 228 visiting the doctor's office) via one or moreaudio sensors (e.g., audio acquisition devices 202, 204, 206, 208, 210,212, 214, 216, 218).

As discussed above, machine vision system 100 may include a plurality ofdiscrete machine vision systems if the monitored space (e.g., monitoredspace 130) within the clinical environment (e.g., the doctor's office)is larger or a higher level of resolution is desired, wherein examplesof machine vision system 100 may include but are not limited to: an RGBimaging system, an infrared imaging system, an ultraviolet imagingsystem, a laser imaging system, a SONAR imaging system, a RADAR imagingsystem, and a thermal imaging system. Accordingly and in certaininstances/embodiments, machine vision system 100 may include one or moreof each of an RGB imaging system, an infrared imaging system, anultraviolet imaging system, a laser imaging system, a SONAR imagingsystem, a RADAR imaging system, and a thermal imaging system positionedthroughout monitored space 130, wherein each of these systems may beconfigured to provide data (e.g., machine vision encounter information102) to ACI compute system 12 and/or modular ACI system 54.

As also discussed above, audio recording system 104 may include aplurality of discrete audio recording systems if the monitored space(e.g., monitored space 130) within the clinical environment (e.g., thedoctor's office) is larger or a higher level of resolution is desired,wherein examples of audio recording system 104 may include but are notlimited to: a handheld microphone, a lapel microphone, an embeddedmicrophone (such as those embedded within eyeglasses, smart phones,tablet computers and/or watches) and an audio recording device.Accordingly and in certain instances/embodiments, audio recording system104 may include one or more of each of a handheld microphone, a lapelmicrophone, an embedded microphone (such as those embedded withineyeglasses, smart phones, tablet computers and/or watches) and an audiorecording device positioned throughout monitored space 130, wherein eachof these microphones/devices may be configured to provide data (e.g.,audio encounter information 106) to ACI compute system 12 and/or modularACI system 54.

Since machine vision system 100 and audio recording system 104 may bepositioned throughout monitored space 130, all of the interactionsbetween medical professionals (e.g., encounter participant 226),patients (e.g., encounter participant 228) and third parties (e.g.,encounter participant 230) that occur during the patient encounter(e.g., encounter participant 228 visiting the doctor's office) withinthe monitored space (e.g., monitored space 130) of the clinicalenvironment (e.g., the doctor's office) may bemonitored/recorded/processed. Accordingly, a patient “check-in” areawithin monitored space 130 may be monitored to obtain encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) during this pre-visit portion of the patientencounter (e.g., encounter participant 228 visiting the doctor'soffice). Further, various rooms within monitored space 130 may bemonitored to obtain encounter information (e.g., machine visionencounter information 102 and/or audio encounter information 106) duringthese various portions of the patient encounter (e.g., while meetingwith the doctor, while vital signs and statistics are obtained, andwhile imaging is performed). Further, a patient “check-out” area withinmonitored space 130 may be monitored to obtain encounter information(e.g., machine vision encounter information 102 and/or audio encounterinformation 106) during this post-visit portion of the patient encounter(e.g., encounter participant 228 visiting the doctor's office).Additionally and via machine vision encounter information 102, visualspeech recognition (via visual lip reading functionality) may beutilized by ambient cooperative intelligence process 10 to furthereffectuate the gathering of audio encounter information 106.

Accordingly and when obtaining 300 encounter information (e.g., machinevision encounter information 102 and/or audio encounter information106), ambient cooperative intelligence process 10 may: obtain 306encounter information (e.g., machine vision encounter information 102and/or audio encounter information 106) from a medical professional(e.g., encounter participant 226); obtain 308 encounter information(e.g., machine vision encounter information 102 and/or audio encounterinformation 106) from a patient (e.g., encounter participant 228);and/or obtain 310 encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) from a thirdparty (e.g., encounter participant 230). Further and when obtaining 300encounter information (e.g., machine vision encounter information 102and/or audio encounter information 106), ambient cooperativeintelligence process 10 may obtain 300 the encounter information (e.g.,machine vision encounter information 102 and/or audio encounterinformation 106) from previous (related or unrelated) patientencounters. For example, if the current patient encounter is actuallythe third visit that the patient is making concerning e.g., shortness ofbreath, the encounter information from the previous two visits (i.e.,the previous two patient encounters) may be highly-related and may beobtained 300 by ambient cooperative intelligence process 10.

When ambient cooperative intelligence process 10 obtains 300 theencounter information, ambient cooperative intelligence process 10 mayutilize 312 a virtual assistant (e.g., virtual assistant 238) to promptthe patient (e.g., encounter participant 228) to provide at least aportion of the encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) during apre-visit portion (e.g., a patient intake portion) of the patientencounter (e.g., encounter participant 228 visiting the doctor'soffice).

Further and when ambient cooperative intelligence process 10 obtains 300encounter information, ambient cooperative intelligence process 10 mayutilize 314 a virtual assistant (e.g., virtual assistant 238) to promptthe patient (e.g., encounter participant 228) to provide at least aportion of the encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) during apost-visit portion (e.g., a patient follow-up portion) of the patientencounter (e.g., encounter participant 228 visiting the doctor'soffice).

Automated Transcript Generation

Ambient cooperative intelligence process 10 may be configured to processthe encounter information (e.g., machine vision encounter information102 and/or audio encounter information 106) to generate encountertranscript 234 that may be automatically formatted and punctuated.

Accordingly and referring also to FIG. 5, ambient cooperativeintelligence process 10 may be configured to obtain 300 encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) of a patient encounter (e.g., a visit to adoctor's office).

Ambient cooperative intelligence process 10 may process 350 theencounter information (e.g., machine vision encounter information 102and/or audio encounter information 106) to: associate a first portion ofthe encounter information (e.g., machine vision encounter information102 and/or audio encounter information 106) with a first encounterparticipant, and associate at least a second portion of the encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) with at least a second encounter participant.

As discussed above, modular ACI system 54 may be configured to form oneor more audio recording beams (e.g., audio recording beams 220, 222,224) via the discrete audio acquisition devices (e.g., discrete audioacquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218)included within audio recording system 104, wherein modular ACI system54 may be further configured to steer the one or more audio recordingbeams (e.g., audio recording beams 220, 222, 224) toward one or moreencounter participants (e.g., encounter participants 226, 228, 230) ofthe above-described patient encounter.

Accordingly and continuing with the above-stated example, modular ACIsystem 54 may steer audio recording beam 220 toward encounterparticipant 226, may steer audio recording beam 222 toward encounterparticipant 228, and may steer audio recording beam 224 toward encounterparticipant 230. Accordingly and due to the directionality of audiorecording beams 220, 222, 224, audio encounter information 106 mayinclude three components, namely audio encounter information 106A (whichis obtained via audio recording beam 220), audio encounter information106B (which is obtained via audio recording beam 222) and audioencounter information 106C (which is obtained via audio recording beam220).

Further and as discussed above, ACI compute system 12 may be configuredto access one or more datasources 118 (e.g., plurality of individualdatasources 120, 122, 124, 126, 128), examples of which may include butare not limited to one or more of a user profile datasource, a voiceprint datasource, a voice characteristics datasource (e.g., for adaptingthe automated speech recognition models), a face print datasource, ahumanoid shape datasource, an utterance identifier datasource, awearable token identifier datasource, an interaction identifierdatasource, a medical conditions symptoms datasource, a prescriptionscompatibility datasource, a medical insurance coverage datasource, aphysical events datasource, and a home healthcare datasource.

Accordingly, ambient cooperative intelligence process 10 may process 350the encounter information (e.g., machine vision encounter information102 and/or audio encounter information 106) to: associate a firstportion (e.g., encounter information 106A) of the encounter information(e.g., audio encounter information 106) with a first encounterparticipant (e.g., encounter participant 226), and associate at least asecond portion (e.g., encounter information 106B, 106C) of the encounterinformation (e.g., audio encounter information 106) with at least asecond encounter participant (e.g., encounter participants 228, 230;respectively).

Further and when processing 350 the encounter information (e.g., audioencounter information 106A, 106B, 106C), ambient cooperativeintelligence process 10 may compare each of audio encounter information106A, 106B, 106C to the voice prints defined within the above-referencedvoice print datasource so that the identity of encounter participants226, 228, 230 (respectively) may be determined. Accordingly, if thevoice print datasource includes a voice print that corresponds to one ormore of the voice of encounter participant 226 (as heard within audioencounter information 106A), the voice of encounter participant 228 (asheard within audio encounter information 106B) or the voice of encounterparticipant 230 (as heard within audio encounter information 106C), theidentity of one or more of encounter participants 226, 228, 230 may bedefined. And in the event that a voice heard within one or more of audioencounter information 106A, audio encounter information 106B or audioencounter information 106C is unidentifiable, that one or moreparticular encounter participant may be defined as “UnknownParticipant”.

Once the voices of encounter participants 226, 228, 230 are processed350, ambient cooperative intelligence process 10 may generate 302 anencounter transcript (e.g., encounter transcript 234) based, at least inpart, upon the first portion of the encounter information (e.g., audioencounter information 106A) and the at least a second portion of theencounter information (e.g., audio encounter information 106B. 106C).

Automated Role Assignment

Ambient cooperative intelligence process 10 may be configured toautomatically define roles for the encounter participants (e.g.,encounter participants 226, 228, 230) in the patient encounter (e.g., avisit to a doctor's office).

Accordingly and referring also to FIG. 6, ambient cooperativeintelligence process 10 may be configured to obtain 300 encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) of a patient encounter (e.g., a visit to adoctor's office).

Ambient cooperative intelligence process 10 may then process 400 theencounter information (e.g., machine vision encounter information 102and/or audio encounter information 106) to associate a first portion ofthe encounter information with a first encounter participant (e.g.,encounter participant 226) and assign 402 a first role to the firstencounter participant (e.g., encounter participant 226).

When processing 400 the encounter information (e.g., machine visionencounter information 102 and/or audio encounter information 106) toassociate the first portion of the encounter information with the firstencounter participant (e.g., encounter participant 226), ambientcooperative intelligence process 10 may process 404 the encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) to associate a first portion of the audioencounter information (e.g., audio encounter information 106A) with thefirst encounter participant (e.g., encounter participant 226).

Specifically and when processing 404 the encounter information (e.g.,machine vision encounter information 102 and/or audio encounterinformation 106) to associate the first portion of the audio encounterinformation (e.g., audio encounter information 106A) with the firstencounter participant (e.g., encounter participant 226), ambientcooperative intelligence process 10 may compare 406 one or more voiceprints (defined within voice print datasource) to one or more voicesdefined within the first portion of the audio encounter information(e.g., audio encounter information 106A); and may compare 408 one ormore utterance identifiers (defined within utterance datasource) to oneor more utterances defined within the first portion of the audioencounter information (e.g., audio encounter information 106A); whereincomparisons 406, 408 may allow ambient cooperative intelligence process10 to assign 402 a first role to the first encounter participant (e.g.,encounter participant 226). For example, if the identity of encounterparticipant 226 can be defined via voice prints, a role for encounterparticipant 226 may be assigned 402 if that identity defined isassociated with a role (e.g., the identity defined for encounterparticipant 226 is Doctor Susan Jones). Further, if an utterance made byencounter participant 226 is “I am Doctor Susan Jones”, this utterancemay allow a role for encounter participant 226 to be assigned 402.

When processing 400 the encounter information (e.g., machine visionencounter information 102 and/or audio encounter information 106) toassociate the first portion of the encounter information with the firstencounter participant (e.g., encounter participant 226), ambientcooperative intelligence process 10 may process 410 the encounterinformation (e.g., machine vision encounter information 102 and/or audioencounter information 106) to associate a first portion of the machinevision encounter information (e.g., machine vision encounter information102A) with the first encounter participant (e.g., encounter participant226).

Specifically and when processing 410 the encounter information (e.g.,machine vision encounter information 102 and/or audio encounterinformation 106) to associate the first portion of the machine visionencounter information (e.g., machine vision encounter information 102A)with the first encounter participant (e.g., encounter participant 226),ambient cooperative intelligence process 10 may compare 412 one or moreface prints (defined within face print datasource) to one or more facesdefined within the first portion of the machine vision encounterinformation (e.g., machine vision encounter information 102A); compare414 one or more wearable token identifiers (defined within wearabletoken identifier datasource) to one or more wearable tokens definedwithin the first portion of the machine vision encounter information(e.g., machine vision encounter information 102A); and compare 416 oneor more interaction identifiers (defined within interaction identifierdatasource) to one or more humanoid interactions defined within thefirst portion of the machine vision encounter information (e.g., machinevision encounter information 102A); wherein comparisons 412, 414, 416may allow ambient cooperative intelligence process 10 to assign 402 afirst role to the first encounter participant (e.g., encounterparticipant 226). For example, if the identity of encounter participant226 can be defined via face prints, a role for encounter participant 226may be assigned 402 if that identity defined is associated with a role(e.g., the identity defined for encounter participant 226 is DoctorSusan Jones). Further, if a wearable token worn by encounter participant226 can be identified as a wearable token assigned to Doctor SusanJones, a role for encounter participant 226 may be assigned 402.Additionally, if an interaction made by encounter participant 226corresponds to the type of interaction that is made by a doctor, theexistence of this interaction may allow a role for encounter participant226 to be assigned 402.

Examples of such wearable tokens may include but are not limited towearable devices that may be worn by the medical professionals when theyare within monitored space 130 (or after they leave monitored space130). For example, these wearable tokens may be worn by medicalprofessionals when e.g., they are moving between monitored rooms withinmonitored space 130, travelling to and/or from monitored space 130,and/or outside of monitored space 130 (e.g., at home).

Additionally, ambient cooperative intelligence process 10 may process418 the encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) to associate atleast a second portion of the encounter information with at least asecond encounter participant; and may assign 420 at least a second roleto the at least a second encounter participant.

Specifically, ambient cooperative intelligence process 10 may process418 the encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) to associate atleast a second portion of the encounter information with at least asecond encounter participant. For example, ambient cooperativeintelligence process 10 may process 418 the encounter information (e.g.,machine vision encounter information 102 and/or audio encounterinformation 106) to associate audio encounter information 106B andmachine vision encounter information 102B with encounter participant 228and may associate audio encounter information 106C and machine visionencounter information 102C with encounter participant 230.

Further, ambient cooperative intelligence process 10 may assign 420 atleast a second role to the at least a second encounter participant. Forexample, ambient cooperative intelligence process 10 may assign 420 arole to encounter participants 228, 230.

Automated Movement Tracking

Ambient cooperative intelligence process 10 may be configured to trackthe movement and/or interaction of humanoid shapes within the monitoredspace (e.g., monitored space 130) during the patient encounter (e.g., avisit to a doctor's office) so that e.g., the ambient cooperativeintelligence process 10 knows when encounter participants (e.g., one ormore of encounter participants 226, 228, 230) enter, exit or cross pathswithin monitored space 130.

Accordingly and referring also to FIG. 7, ambient cooperativeintelligence process 10 may process 450 the machine vision encounterinformation (e.g., machine vision encounter information 102) to identifyone or more humanoid shapes. As discussed above, examples of machinevision system 100 generally (and ACI client electronic device 34specifically) may include but are not limited to one or more of an RGBimaging system, an infrared imaging system, an ultraviolet imagingsystem, a laser imaging system, a SONAR imaging system, a RADAR imagingsystem, and a thermal imaging system).

When ACI client electronic device 34 includes a visible light imagingsystem (e.g., an RGB imaging system), ACI client electronic device 34may be configured to monitor various objects within monitored space 130by recording motion video in the visible light spectrum of these variousobjects. When ACI client electronic device 34 includes an invisiblelight imaging system (e.g., a laser imaging system, an infrared imagingsystem and/or an ultraviolet imaging system), ACI client electronicdevice 34 may be configured to monitor various objects within monitoredspace 130 by recording motion video in the invisible light spectrum ofthese various objects. When ACI client electronic device 34 includes anX-ray imaging system, ACI client electronic device 34 may be configuredto monitor various objects within monitored space 130 by recordingenergy in the X-ray spectrum of these various objects. When ACI clientelectronic device 34 includes a SONAR imaging system, ACI clientelectronic device 34 may be configured to monitor various objects withinmonitored space 130 by transmitting soundwaves that may be reflected offof these various objects. When ACI client electronic device 34 includesa RADAR imaging system, ACI client electronic device 34 may beconfigured to monitor various objects within monitored space 130 bytransmitting radio waves that may be reflected off of these variousobjects. When ACI client electronic device 34 includes a thermal imagingsystem, ACI client electronic device 34 may be configured to monitorvarious objects within monitored space 130 by tracking the thermalenergy of these various objects.

As discussed above, ACI compute system 12 may be configured to accessone or more datasources 118 (e.g., plurality of individual datasources120, 122, 124, 126, 128), wherein examples of which may include but arenot limited to one or more of a user profile datasource, a voice printdatasource, a voice characteristics datasource (e.g., for adapting theautomated speech recognition models), a face print datasource, ahumanoid shape datasource, an utterance identifier datasource, awearable token identifier datasource, an interaction identifierdatasource, a medical conditions symptoms datasource, a prescriptionscompatibility datasource, a medical insurance coverage datasource, aphysical events datasource, and a home healthcare datasource.

Accordingly and when processing 450 the machine vision encounterinformation (e.g., machine vision encounter information 102) to identifyone or more humanoid shapes, ambient cooperative intelligence process 10may be configured to compare the humanoid shapes defined within one ormore datasources 118 to potential humanoid shapes within the machinevision encounter information (e.g., machine vision encounter information102).

When processing 450 the machine vision encounter information (e.g.,machine vision encounter information 102) to identify one or morehumanoid shapes, ambient cooperative intelligence process 10 may track452 the movement of the one or more humanoid shapes within the monitoredspace (e.g., monitored space 130). For example and when tracking 452 themovement of the one or more humanoid shapes within monitored space 130,ambient cooperative intelligence process 10 may add 454 a new humanoidshape to the one or more humanoid shapes when the new humanoid shapeenters the monitored space (e.g., monitored space 130) and/or may remove456 an existing humanoid shape from the one or more humanoid shapes whenthe existing humanoid shape leaves the monitored space (e.g., monitoredspace 130).

For example, assume that a lab technician (e.g., encounter participant242) temporarily enters monitored space 130 to chat with encounterparticipant 230. Accordingly, ambient cooperative intelligence process10 may add 454 encounter participant 242 to the one or more humanoidshapes being tracked 452 when the new humanoid shape (i.e., encounterparticipant 242) enters monitored space 130. Further, assume that thelab technician (e.g., encounter participant 242) leaves monitored space130 after chatting with encounter participant 230. Therefore, ambientcooperative intelligence process 10 may remove 456 encounter participant242 from the one or more humanoid shapes being tracked 452 when thehumanoid shape (i.e., encounter participant 242) leaves monitored space130.

Also and when tracking 452 the movement of the one or more humanoidshapes within monitored space 130, ambient cooperative intelligenceprocess 10 may monitor the trajectories of the various humanoid shapeswithin monitored space 130. Accordingly, assume that when leavingmonitored space 130, encounter participant 242 walks in front of (orbehind) encounter participant 226. As ambient cooperative intelligenceprocess 10 is monitoring the trajectories of (in this example) encounterparticipant 242 (who is e.g., moving from left to right) and encounterparticipant 226 (who is e.g., stationary), when encounter participant242 passes in front of (or behind) encounter participant 226, theidentities of these two humanoid shapes may not be confused by ambientcooperative intelligence process 10.

Ambient cooperative intelligence process 10 may be configured to obtain300 the encounter information of the patient encounter (e.g., a visit toa doctor's office), which may include machine vision encounterinformation 102 (in the manner described above) and/or audio encounterinformation 106.

Ambient cooperative intelligence process 10 may steer 458 one or moreaudio recording beams (e.g., audio recording beams 220, 222, 224) towardthe one or more humanoid shapes (e.g., encounter participants 226, 228,230) to capture audio encounter information (e.g., audio encounterinformation 106), wherein audio encounter information 106 may beincluded within the encounter information (e.g., machine visionencounter information 102 and/or audio encounter information 106).

Specifically and as discussed above, ambient cooperative intelligenceprocess 10 (via modular ACI system 54 and/or audio recording system 104)may utilize one or more of the discrete audio acquisition devices (e.g.,audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218)to form an audio recording beam. For example, modular ACI system 54and/or audio recording system 104 may be configured to utilize variousaudio acquisition devices to form audio recording beam 220, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 226 (as audio recording beam 220 is pointed to (i.e.,directed toward) encounter participant 226). Additionally, modular ACIsystem 54 and/or audio recording system 104 may be configured to utilizevarious audio acquisition devices to form audio recording beam 222, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 228 (as audio recording beam 222 is pointed to (i.e.,directed toward) encounter participant 228). Additionally, modular ACIsystem 54 and/or audio recording system 104 may be configured to utilizevarious audio acquisition devices to form audio recording beam 224, thusenabling the capturing of audio (e.g., speech) produced by encounterparticipant 230 (as audio recording beam 224 is pointed to (i.e.,directed toward) encounter participant 230).

Once obtained, ambient cooperative intelligence process 10 may process302 the encounter information (e.g., machine vision encounterinformation 102 and/or audio encounter information 106) to generateencounter transcript 234 and may process 304 at least a portion ofencounter transcript 234 to populate at least a portion of a medicalrecord (e.g., medical record 236) associated with the patient encounter(e.g., a visit to a doctor's office).

Fully-Autonomous/Semi-Autonomous Scanning:

As discussed above and as shown in FIG. 2, modular ACI system 54 may beconfigured to automate cooperative intelligence, wherein modular ACIsystem 54 may include: machine vision system 100 configured to obtainmachine vision encounter information 102 concerning a patient encounter;audio recording system 104 configured to obtain audio encounterinformation 106 concerning the patient encounter; and a compute system(e.g., ACI compute system 12) configured to receive machine visionencounter information 102 and audio encounter information 106 frommachine vision system 100 and audio recording system 104 (respectively).

As also discussed above, machine vision system 100 may include but isnot limited to: one or more ACI client electronic devices (e.g., ACIclient electronic device 34, examples of which may include but are notlimited to an RGB imaging system, an infrared imaging system, aultraviolet imaging system, a laser imaging system, a SONAR imagingsystem, a RADAR imaging system, and a thermal imaging system).

As also discussed above, audio recording system 104 may include but isnot limited to: one or more ACI client electronic devices (e.g., ACIclient electronic device 30, examples of which may include but are notlimited to a handheld microphone (e.g., one example of a body wornmicrophone), a lapel microphone (e.g., another example of a body wornmicrophone), an embedded microphone, such as those embedded withineyeglasses, smart phones, tablet computers and/or watches (e.g., anotherexample of a body worn microphone), and an audio recording device).

Further and as shown in FIG. 3, machine vision system 100 and audiorecording system 104 may be combined into one package to formmixed-media ACI device 232. For example, mixed-media ACI device 232 maybe configured to be mounted to a structure (e.g., a wall, a ceiling, abeam, a column) within the above-described clinical environments (e.g.,a doctor's office, a medical facility, a medical practice, a medicallab, an urgent care facility, a medical clinic, an emergency room, anoperating room, a hospital, a long term care facility, a rehabilitationfacility, a nursing home, and a hospice facility), thus allowing foreasy installation of the same.

Modular ACI system 54 may be further configured to steer the one or moreaudio recording beams (e.g., audio recording beams 220, 222, 224) towardone or more encounter participants (e.g., encounter participants 226,228, 230) of the patient encounter based, at least in part, upon machinevision encounter information 102. As discussed above, mixed-media ACIdevice 232 (and machine vision system 100/audio recording system 104included therein) may be configured to monitor one or more encounterparticipants (e.g., encounter participants 226, 228, 230) of a patientencounter.

Specifically and as discussed above, machine vision system 100 (eitheras a stand-alone system or as a component of mixed-media ACI device 232)may be configured to detect humanoid shapes within the above-describedclinical environments (e.g., a doctor's office, a medical facility, amedical practice, a medical lab, an urgent care facility, a medicalclinic, an emergency room, an operating room, a hospital, a long termcare facility, a rehabilitation facility, a nursing home, and a hospicefacility). And when these humanoid shapes are detected by machine visionsystem 100, modular ACI system 54 and/or audio recording system 104 maybe configured to utilize one or more of the discrete audio acquisitiondevices (e.g., audio acquisition devices 202, 204, 206, 208, 210, 212,214, 216, 218) to form an audio recording beam (e.g., audio recordingbeams 220, 222, 224) that is directed toward each of the detectedhumanoid shapes (e.g., encounter participants 226, 228, 230).

Accordingly, it is foreseeable that one of more of these systems/devicesinclude within modular ACI system 54 (e.g., machine vision system 100,audio recording system 104, mixed-media ACI device 232, and/or audioacquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) mayneed to be calibrated (e.g., initially calibrated and/or subsequentlyrecalibrated).

Referring also to FIGS. 8-9, such calibration of these one of moresystems/devices included within modular ACI system 54 (e.g., machinevision system 100, audio recording system 104, mixed-media ACI device232, and/or audio acquisition devices 202, 204, 206, 208, 210, 212, 214,216, 218) may be effectuated via ACI calibration platform 500.

ACI calibration platform 500 may include video recording subsystem 502configured to generate 550 a three-dimensional model (e.g.,three-dimensional model 504) of at least a portion of athree-dimensional space (e.g., monitored space 130) incorporating an ACIsystem (e.g., modular ACI system 54). ACI calibration platform 500(generally) and video recording subsystem 502 (specifically) may include(or be interfaced with) machine vision technology, examples of which mayinclude but are not limited to: an RGB imaging system, an infraredimaging system, an ultraviolet imaging system, a laser imaging system, aSONAR imaging system, a RADAR imaging system, and a thermal imagingsystem.

As discussed above, an example of monitored space 130 may include but isnot limited to a clinical environment (e.g., a doctor's office, amedical facility, a medical practice, a medical lab, an urgent carefacility, a medical clinic, an emergency room, an operating room, ahospital, a long term care facility, a rehabilitation facility, anursing home, and a hospice facility).

This three-dimensional model (e.g., three-dimensional model 504)generated by video recording subsystem 502 of ACI calibration platform500 may be configured to define one or more:

-   -   Subspaces: One or more subspaces within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this subspace        may include but is not limited to visitor waiting space 506        (which is shown to include encounter participants 230, 242).    -   Objects: One or more objects within the three-dimensional space        (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein examples of these objects        may include but are not limited to: physician desk 508 and        examination table 510.    -   Features: One or more features within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this feature        may include but is not limited to: window 512.    -   Interaction Zones: One or more interaction zones within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this interaction zone may include but is not limited to:        examination zone 514 (i.e., an area proximate examination table        510).    -   Noise Sources: One or more noise sources within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this noise source may include but is not limited to: HVAC        supply air vent 516.

ACI calibration platform 500 may be wirelessly coupled to one or moreexternal systems (e.g., modular ACI system 54) and/or one or moreexternal resources (e.g., one or more of datasources 120, 122, 124, 126,128), thus enabling the transfer of data between ACI calibrationplatform 500 and these external resources and/or datasources.

Accordingly and through such wireless connectivity, three-dimensionalmodel 504 may be wirelessly transferred from ACI calibration platform500 to modular ACI system 54 for processing, which will be discussedbelow in greater detail. Alternatively, three-dimensional model 504 maybe transferred from ACI calibration platform 500 to modular ACI system54 via wired transfer methodologies (e.g., a USB drive; not shown) forprocessing, which will be discussed below in greater detail.

Video recording system 502 may be configured to interface with an objectdatasource (e.g., object datasource 518) that may define a plurality ofobjects that may be located within the three-dimensional space (e.g.,monitored space 130). For example, object datasource 518 may define whata desk “looks” like, and what an examination table “looks” like, andwhat an HVAC vent “looks” like, and what a window “looks” like. Thisfunctionality may be accomplished in a fashion similar to the manner inwhich a facial recognition system knows what a face “looks” like.Depending upon the manner in which ACI calibration platform 500 isconfigured, object datasource 518 may be a locally-accessible datasourcethat is resident on ACI calibration platform 500. Alternatively, objectdatasource 518 may be a remotely-accessible datasource that is residenton modular ACI system 54.

Accordingly and through the use of object datasource 518, ACIcalibration platform 500 may produce a three-dimensional model (e.g.,three-dimensional model 504) in which the objects included/definedtherein may be of a known type (e.g., physician desk 508, examinationtable 510, window 512, HVAC supply air vent 516), which may beaccomplished via tagging/metadata.

ACI calibration platform 500 may include audio generation subsystem 520configured to generate 552 one or more audio calibration signals (e.g.,audio calibration signal 522) for receipt by an audio recording system(e.g., audio recording system 104) included within the ACI system (e.g.,modular ACI system 54). ACI calibration platform 500 (generally) andaudio generation subsystem 520 (specifically) may include (or beinterfaced with) audio rendering technology, an example of which mayinclude but is not limited to a speaker assembly.

The one or more audio calibration signals (e.g., audio calibrationsignal 522) may include one or more of:

-   -   Noise Signal: An example of audio calibration signal 522 may        include but is not limited to: a white noise signal. As is known        in the art, a white noise signal is a random signal having equal        intensity at all frequencies, giving it a constant power        spectral density. The term is used, with this or similar        meanings, in many scientific and technical disciplines,        including physics, acoustical engineering, telecommunications,        and statistical forecasting. White noise refers to a statistical        model for signals and signal sources, rather than to any        specific signal.    -   Sinusoid Signal: An example of audio calibration signal 522 may        include but is not limited to: a sinusoid. A sinusoid signal is        a signal fully characterized by a mathematical function that        describes a smooth periodic oscillation having a fixed        frequency. It is named after the function sine. Sinusoids often        occur in pure and applied mathematics, as well as physics,        engineering, and signal processing.    -   Multi-Frequency Signal: An example of audio calibration signal        522 may include but is not limited to: a sweeping sinusoid. A        sweeping sinusoid signal is a signal fully characterized by a        mathematical function that describes a smooth periodic        oscillation having a frequency that varies with time (typically        between two frequencies, such as a logarithmic sweep from 20 Hz        to 20 kHz in acoustic applications).    -   Impulse Function: An example of audio calibration signal 522 may        include but is not limited to: an impulse function. An impulse        function is a function that is zero everywhere but at the        origin, where the amplitude is infinitely high.

ACI calibration platform 500 may include mobile base assembly 524configured to autonomously position 554 ACI calibration platform 500within the three-dimensional space (e.g., monitored space 130).Accordingly, ACI calibration platform 500 may be configured to move inan automated and controlled fashion within monitored space 130 (e.g., ina fashion similar to that of a robotic autonomous vacuum). For example,ACI calibration platform 500 may include the above-described machinevision technology to enable ACI calibration platform 500 to navigatethrough monitored space 130 via the use of mobile base assembly 524.Additionally, ACI calibration platform 500 (generally) and mobile baseassembly 524 (specifically) may include one of more impact sensors(e.g., impact sensors 526, 528) that sense impact with various objectswithin the monitored space 130 (e.g., walls, doors, furniture) so that,upon sensing such an impact, the direction in which ACI calibrationplatform 500 in travelling may be adjusted (e.g., reversed).

ACI calibration platform 500 may include cleaning assembly 530configured to autonomously clean 556 at least a portion of thethree-dimensional space (e.g., monitored space 130). Examples ofcleaning assembly 530 may include:

-   -   Vacuum Assembly: For example, cleaning assembly 530 may be        configured to vacuum the floor of monitored space 130.    -   Mop Assembly: For example, cleaning assembly 530 may be        configured to mop the floor of monitored space 130.    -   Sterilizing Assembly: For example, cleaning assembly 530 may be        configured to sterilize the floor of monitored space 130 via        e.g., steam generation or ultraviolet light.

While ACI calibration platform 500 is described above as being capableof autonomously moving within monitored space 130 (via mobile baseassembly 524), this is for illustrative purpose only and is not intendedto be a limitation of this disclosure, as other configurations arepossible and are considered to be within the scope of this disclosure.For example, ACI calibration platform 500 may be configured to bemanually positioned within the three-dimensional space (e.g., monitoredspace 130). Therefore, ACI calibration platform 500 may be includedwithin (or a portion of) a handheld client electronic device (such as asmart telephone or a tablet computer).

As is known in the art, such client electronic devices typically includemachine vision technology (such as visible light cameras) and audiorendering technology (such as one or more speaker assemblies) thatenable the generation 550, 552 of three-dimensional model 504 and/oraudio calibration signal 522 In such an implementation, ACI calibrationplatform 500 may be manually manipulated (i.e., moved/positioned) withinmonitored space 130 by a user, wherein the user may move ACI calibrationplatform 500 within monitored space 130 to generate 550three-dimensional model 504 and/or generate 552 audio calibration signal522 from all of the appropriate/required positions within monitoredspace 130.

ACI System Calibration:

As discussed above, ACI calibration platform 500 may be wirelesslycoupled to one or more external systems (e.g., modular ACI system 54)and/or one or more external resources (e.g., one or more of datasources120, 122, 124, 126, 128), thus enabling the transfer of data between ACIcalibration platform 500 and these external resources and/ordatasources. Accordingly and through such wireless connectivity,three-dimensional model 504 may be wirelessly transferred from ACIcalibration platform 500 to modular ACI system 54 for processing.Alternatively, three-dimensional model 504 may be transferred from ACIcalibration platform 500 to modular ACI system 54 for processing vianon-wireless transfer methodologies, such as via a portable datatransfer device (e.g., a USB drive; not shown).

Accordingly, ambient cooperative intelligence process 10 may obtain 600calibration information (e.g., calibration information 532) for athree-dimensional space (e.g., monitored space 130) incorporating an ACIsystem (e.g., modular ACI system 54). As discussed above, thiscalibration information may be obtained from an ACI calibration platform(e.g., ACI calibration platform 500). This calibration information(e.g., calibration information 532) may include three-dimensional model504 and one or more audio calibration signals (e.g., audio calibrationsignal 522).

As discussed above, ACI calibration platform 500 may generatethree-dimensional model 504 for at least a portion of monitored space130 incorporating an ACI system (e.g., modular ACI system 54), whereinthree-dimensional model 504 may be configured to define one or more:

-   -   Subspaces: One or more subspaces within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this subspace        may include but is not limited to visitor waiting space 506        (which is shown to include encounter participants 230, 242).    -   Objects: One or more objects within the three-dimensional space        (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein examples of these objects        may include but are not limited to: physician desk 508 and        examination table 510.    -   Features: One or more features within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this feature        may include but is not limited to: window 512.    -   Interaction Zones: One or more interaction zones within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this interaction zone may include but is not limited to:        examination zone 514 (i.e., an area proximate examination table        510).    -   Noise Sources: One or more noise sources within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this noise source may include but is not limited to: HVAC        supply air vent 516.

As discussed above, these one or more audio calibration signals (e.g.,audio calibration signal 522) may include one or more of:

-   -   Noise Signal: An example of audio calibration signal 522 may        include but is not limited to: a white noise signal. As is known        in the art, a white noise signal is a random signal having equal        intensity at all frequencies, giving it a constant power        spectral density. The term is used, with this or similar        meanings, in many scientific and technical disciplines,        including physics, acoustical engineering, telecommunications,        and statistical forecasting. White noise refers to a statistical        model for signals and signal sources, rather than to any        specific signal.    -   Sinusoid Signal: An example of audio calibration signal 522 may        include but is not limited to: a sinusoid. A sinusoid signal is        a signal fully characterized by a mathematical function that        describes a smooth periodic oscillation having a fixed        frequency. It is named after the function sine. Sinusoids often        occur in pure and applied mathematics, as well as physics,        engineering, and signal processing.    -   Multi-Frequency Signal: An example of audio calibration signal        522 may include but is not limited to: a sweeping sinusoid. A        sweeping sinusoid signal is a signal fully characterized by a        mathematical function that describes a smooth periodic        oscillation having a frequency that varies with time (typically        between two frequencies, such as a logarithmic sweep from 20 Hz        to 20 kHz in acoustic applications).    -   Impulse Function: An example of audio calibration signal 522 may        include but is not limited to: an impulse function. An impulse        function is a function that is zero everywhere but at the        origin, where the amplitude is infinitely high.

Specifically, three-dimensional model 504 may be wirelessly transferred(or transferred via wired transfer) from ACI calibration platform 500 tomodular ACI system 54 in e.g., the manners described above.Additionally, the one or more audio calibration signals (e.g., audiocalibration signal 522) may be acoustically transferred from ACIcalibration platform 500 to modular ACI system 54. For example, audiocalibration signal 522 may be rendered via e.g., a speaker assemblyincluded within ACI calibration platform 500, wherein audio calibrationsignal 522 may be acoustically transferred through the air of monitoredspace 130 and “heard” by modular ACI system 54 via e.g., audioacquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218).

Once the calibration information (e.g., calibration information 532) isobtained 600, ambient cooperative intelligence process 10 may process602 this calibration information (e.g., calibration information 532) tocalibrate (e.g., either initially or subsequently) the ACI system (e.g.,modular ACI system 54).

Specifically, the one or more audio calibration signals (e.g., audiocalibration signal 522) may be utilized, in whole or in part, tocalibrate one or more audio acquisition devices (e.g., audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218) within thethree-dimensional space (e.g., monitored space 130).

As is known in the art, the performance of audio acquisition devices(e.g., audio acquisition devices 202, 204, 206, 208, 210, 212, 214, 216,218) may vary over time. Naturally, such audio acquisition devices maytotally fail and simply not work anymore (which is comparatively simpleto detect). However, such audio acquisition devices may not completelyfail and may only suffer from performance drift where the ability of anolder audio acquisition device to detect e.g., higher frequency signalsis diminished (in a fashion similar to the manner in which the hearingof a human declines as they get older). Accordingly and through the useof the one or more audio calibration signals (e.g., audio calibrationsignal 522), the performance of audio acquisition devices (e.g., audioacquisition devices 202, 204, 206, 208, 210, 212, 214, 216, 218) may bedetermined/measured and compensated (if need be).

As discussed above, modular ACI system 54 may be configured to steer oneor more audio recording beams (e.g., audio recording beams 220, 222,224) toward one or more encounter participants (e.g., encounterparticipants 226, 228, 230) of the above-described patient encounter,wherein modular ACI system 54 may be configured to utilize one or moreof the discrete audio acquisition devices (e.g., audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218) to form these audiorecording beams (e.g., audio recording beams 220, 222, 224).

As discussed above, an example of audio calibration signal 522 may be awhite noise signal (e.g., a random signal having equal intensity at allfrequencies). Accordingly and as ACI calibration platform 500 is moved(and continuously repositioned) within monitored space 130, audiogeneration subsystem 520 within ACI calibration platform 500 may renderaudio calibration signal 522 that is (in this example) a white noisesignal having equal spectral intensity from 20 Hz to 20 kHz. Furtherassume that audio acquisition devices 202, 204, 206, 208, 210, 212, 214,216, 218 are designed to have a flat frequency response (i.e., beequally sensitive) to signals in the range of 100 Hz to 5 kHz.Accordingly and in order to test the performance of audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218, ACI calibrationplatform 500 may move (or be moved) to a sufficient number of locationswithin monitored space 130 to ensure spatial coverage of the acousticenvironment (e.g., by measuring a number of acoustic paths from typicalinteraction zones to the various ACI devices (e.g., audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218) within monitoredspace 130).

Since (in this example) audio calibration signal 522 is a white noisesignal having equal spectral intensity from 20 Hz to 20 kHz, thefrequency response of each of audio acquisition devices 202, 204, 206,208, 210, 212, 214, 216, 218 from 100 Hz to 5 kHz should be flat (i.e.,have equal intensity). In the event that one of audio acquisitiondevices 202, 204, 206, 208, 210, 212, 214, 216, 218 is not producing anysignal, this particular audio acquisition device may have failed and maylikely need replacement.

And in the event that one of audio acquisition devices 202, 204, 206,208, 210, 212, 214, 216, 218 is performing oddly, this particular audioacquisition device may need to be compensated. For example:

-   -   if audio acquisition device 202 is over-sensitive at 1 kz and is        producing an output signal that is 6 db above where it is        expected to be @ 1 kHz, ambient cooperative intelligence process        10 may attenuate the output signal provided by audio acquisition        device 202 by a factor of 6 db @ 1 kHz.    -   if audio acquisition device 206 is under-sensitive at 3 kz and        is producing an output signal that is 8 db below where it is        expected to be @ 3 kHz, ambient cooperative intelligence process        10 may be configured to amplify the output signal provided by        audio acquisition device 206 by a factor of 8 db @ 3 kHz.

Further, the three-dimensional model (e.g., three-dimensional model 504)may be utilized, in whole or in part, to steer one or more audiorecording beams (e.g., audio recording beams 220, 222, 224) within thethree-dimensional space (e.g., monitored space 130).

As discussed above and through the use of object datasource 518, ACIcalibration platform 500 may produce a three-dimensional model (e.g.,three-dimensional model 504) in which the objects included/definedtherein may be of a known type (e.g., physician desk 508, examinationtable 510, window 512, HVAC supply air vent 516), which may beaccomplished via tagging/metadata. Accordingly, ambient cooperativeintelligence process 10 may determine acoustic propagation channelinformation for the purpose of e.g., robust automated speechrecognition, signal enhancement, audio recording beam forming, acousticecho cancellation, null steering and blind source separation. Thisacoustic path information may be associated with the spatial informationdefined within three-dimensional model 504.

As is known in the art, echo cancellation is a method for improvingsignal quality by removing echo after it is already present. This methodmay be called acoustic echo suppression (AES) and acoustic echocancellation (AEC), and more rarely and in context of telecommunicationsline echo cancellation (LEC). In some cases, these terms are moreprecise, as there are various types and causes of echo with uniquecharacteristics, including acoustic echo (sounds from a loudspeakerbeing reflected, coupled to and recorded by a microphone, which can varysubstantially over time) and line echo (electrical echo signals causedby e.g., coupling between the sending and receiving wires, impedancemismatches, electrical reflections, etc., which varies much less thanacoustic echo). Accordingly and in this configuration, such echocancellation methodologies may be utilized to e.g., eliminate the couplesignal of a second speaker that appears in the audio recording beamsteered at a closely-positioned first speaker; while also eliminatingthe coupled signal of the first speaker that appears in the audiorecording beam steered at the closely-positioned second speaker.

As is known in the art, null-steering precoding is a method of spatialsignal processing by which a multiple antenna transmitter may nullmultiuser interference signals in wireless communications, whereinnull-steering precoding may mitigate the impact of background noise andunknown user interference. In particular, null-steering precoding may bea method of beamforming for narrowband signals that may compensate fordelays of receiving signals from a specific source at different elementsof an antenna array. In general and to improve performance of theantenna array, incoming signals may be summed and averaged, whereincertain signals may be weighted and compensation may be made for signaldelays.

As is known in the art, blind source separation is the separation of aset of source signals from a set of mixed signals, without the aid ofinformation (or with very little information) about the source signalsor the mixing process. Since the chief difficulty of blind sourceseparation is its underdetermination, methods for blind sourceseparation generally seek to narrow the set of possible solutions in away that is unlikely to exclude the desired solution. In one approach,exemplified by principal and independent component analysis, one seekssource signals that are minimally correlated or maximally independent ina probabilistic or information-theoretic sense. A second approach,exemplified by nonnegative matrix factorization, is to impose structuralconstraints on the source signals.

As discussed above, three-dimensional model 504 may be configured todefine one or more:

-   -   Subspaces: One or more subspaces within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this subspace        may include but is not limited to visitor waiting space 506        (which is shown to include encounter participants 230, 242).    -   Objects: One or more objects within the three-dimensional space        (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein examples of these objects        may include but are not limited to: physician desk 508 and        examination table 510.    -   Features: One or more features within the three-dimensional        space (e.g., monitored space 130) may be defined within        three-dimensional model 504, wherein an example of this feature        may include but is not limited to: window 512.    -   Interaction Zones: One or more interaction zones within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this interaction zone may include but is not limited to:        examination zone 514 (i.e., an area proximate examination table        510).    -   Noise Sources: One or more noise sources within the        three-dimensional space (e.g., monitored space 130) may be        defined within three-dimensional model 504, wherein an example        of this noise source may include but is not limited to: HVAC        supply air vent 516.

Each of the specific subspaces, objects, features, interaction zones andnoise sources defined within three-dimensional model 504 may havepositive/negative impact concerning the manner in which audio recordingbeams (e.g., audio recording beams 220, 222, 224) may be steered withinthe three-dimensional space (e.g., monitored space 130).

For example and with respect to visitor waiting space 506 (which isshown to include encounter participants 230, 242), ambient cooperativeintelligence process 10 may disfavor steering audio recording beams(e.g., audio recording beams 220, 222, 224) toward visitor waiting space506, as this is a waiting area and it is less likely that substantiveinformation may be extracted from conversations that are occurring inthis area.

Conversely and with respect to physician desk 508 and examination table510, ambient cooperative intelligence process 10 may favor steeringaudio recording beams (e.g., audio recording beams 220, 222, 224) towardphysician desk 508 and examination table 510, as it is more likely thatsubstantive information may be extracted from conversations that areoccurring in these areas.

With respect to window 512, ambient cooperative intelligence process 10may disfavor steering audio recording beams (e.g., audio recording beams220, 222, 224) toward window 512, as this is a hard surface and it ismore likely that high levels of reflection/echo/noise may be includedwithin information captured with an audio recording beam directed towardwindow 512.

Conversely and with respect to examination zone 514 (i.e., an areaproximate examination table 510), ambient cooperative intelligenceprocess 10 may favor steering audio recording beams (e.g., audiorecording beams 220, 222, 224) toward examination zone 514, as it ismore likely that substantive information may be extracted fromconversations that are occurring in this area.

From the above-described calibration step, a model that locates objectswithin the three-dimensional space (e.g., monitored space 130) may bedefined, wherein this model may aid the natural language understandingfunctionality of ambient cooperative intelligence process 10. Forexample, if ambient cooperative intelligence process 10 identifies soundcoming from a location that corresponds to an examination bed, thisinformation may be very useful for the natural language understandingfunctionality of ambient cooperative intelligence process 10,particularly if modular ACI system 54 does not include machine vision.

With respect to HVAC supply air vent 516, ambient cooperativeintelligence process 10 may disfavor steering audio recording beams(e.g., audio recording beams 220, 222, 224) toward HVAC supply air vent516, as this is a noisy object and it is more likely that high levels ofnoise may be included within information captured with an audiorecording beam directed toward HVAC supply air vent 516.

Non-Medical Applications:

As discussed above, while ambient cooperative intelligence process 10was described above as being utilized to automate the collection andprocessing of clinical encounter information togenerate/store/distribute medical records, this is for illustrativepurposes only and is not intended to be a limitation of this disclosure,as other configurations are possible and are considered to be within thescope of this disclosure. Accordingly, such encounter information mayinclude but are not limited to the following examples.

Financial Information:

For example, ambient cooperative intelligence process 10 generally(and/or ACD system 54 specifically) may be configured to automate thecollection and processing of financial data that is generated during anencounter in which financial information is discussed. An example ofsuch an encounter may include but is not limited to a meeting between anindividual and a financial advisor. For example, ambient cooperativeintelligence process 10 may be configured to supplement/complement afinancial advisor's knowledge by recommending products, answeringquestions and making offers based on the conversation that the financialadvisor is having with a client in essentially real time, as well ascompleting various forms, mortgage applications, stock purchase/saleorders, estate planning documents, etc.

Benefits: The benefits achievable by ambient cooperative intelligenceprocess 10 when configured to process financial information may beconsiderable. For example and as is understandable, financial advisorsmay not know all things concerning e.g., financial and investmentinstruments. Accordingly, ambient cooperative intelligence process 10(when configured to process financial information) may monitor aconversation between the financial advisor and the client. Ambientcooperative intelligence process 10 may then utilize natural languageprocessing and artificial intelligence to identify issues/questionswithin the conversation and leverage collective knowledge to providepertinent information to the financial advisor.

For example, assume that a client visits a financial advisor seekingfinancial advice concerning tax free/tax deferred retirement savings.Accordingly and through the use of the various systems described above(e.g., audio input device 30, display device 32, machine vision inputdevice 34, and audio rendering device 116), ambient cooperativeintelligence process 10 (when configured to process financialinformation) may monitor the conversation between the financial advisorand the client. Assuming that this is the first time that this client ismeeting with his financial advisor, the information obtained during thisinitial meeting may be parsed and used to populate the various fields ofa client intake form. For example, the client may identify themself andtheir name may be entered into the client intake form. Additionally,ambient cooperative intelligence process 10 may be configured to definea voiceprint and/or face print for the client so that e.g. in the futurethis voiceprint and/or face print may be utilized to authenticate theclient's identity when they want to access their data. Additionally,when the client identifies e.g. their age, their marital status, theirspouse's name, their spouse's age, and whether or not they have childrenand (if so) the age of their children, all of this information may beused to populate this client intake form.

Continuing with the above stated example, assume that the client asksabout tax-free/tax-deferred retirement savings plans. The financialadvisor may then ask them what their income was last year. As ambientcooperative intelligence process 10 may be monitoring this conversationvia audio input device 30, ambient cooperative intelligence process 10may “hear” that the client is interested in tax-free/tax-deferredretirement savings plans and what their income level is. Accordingly andthrough the use of the above-described natural language processing andartificial intelligence, ambient cooperative intelligence process 10 maydetermine whether or not the client qualifies for a 401(k) retirementplan, a pre-tax/post-tax traditional IRA plan, and/or a pre-tax/post-taxRoth IRA plan. Upon making such a determination, ambient cooperativeintelligence process 10 may provide supplemental information to thefinancial advisor so that the financial advisor may provide guidance tothe client.

For example, ambient cooperative intelligence process 10 may render (ondisplay device 32) a list of the tax-free/tax-deferred retirementsavings plans for which the client qualifies.Additionally/alternatively, this information may be audibly rendered(e.g. covertly into an earbud worn by the financial advisor) so that thefinancial advisor may provide such information to the client.

Accordingly and through the use of such a system, ambient cooperativeintelligence process 10 (when configured to process financialinformation) may monitor the conversation between (in this example) thefinancial advisor and a client to e.g. gather information and populateclient intake forms, generate voice prints and/or face prints for clientauthentication, listen to inquiries made by the client, and provideresponses to those inquiries so that the financial advisor may provideguidance to the client.

Additionally, ambient cooperative intelligence process 10 may beconfigured to monitor the advice that the financial advisor is providingto the client and confirm the accuracy of the same, wherein covertcorrections/notifications may be provided to the financial advisor inthe event that the financial advisor misspoke (e.g., advising the clientthat they qualify for a retirement plan when they actually do notqualify).

Further, ambient cooperative intelligence process 10 may be configuredto provide guidance to the financial advisor/client even when suchguidance is not sought. For example, if this client said that they havechildren, ambient cooperative intelligence process 10 may prompt thefinancial advisor to inquire as to what college savings plans (e.g.529s) they have in place for their children. And if none are in place,the financial advisor may be prompted to explain the tax benefits ofsuch plans.

Further still, ambient cooperative intelligence process 10 may beconfigured to covertly provide information to the financial advisor thatmay assist in building a relationship between the financial advisor andclient. For example, assume that the client explained that his wife'sname was Jill (during the first meeting between the client and thefinancial advisor) and the client explained that he and his wife weregoing to be visiting Italy over the summer. Assume that the clientreturns to meet with the financial advisor in the fall. During the firstvisit, ambient cooperative intelligence process 10 may (as discussedabove) populate a client intake form that identifies the client spouseas Jill. Further, ambient cooperative intelligence process 10 may make anote that the client and Jill are going to be visiting Italy in thesummer of 2020. Assuming that this follow-up meeting is after the summerof 2020, ambient cooperative intelligence process 10 may covertly promptthe financial advisor to ask the client if he and Jill enjoyed Italy,thus enabling the establishment of goodwill between the client and thefinancial advisor.

Ambient cooperative intelligence process 10 may further be configured toauto-populate forms that may be required based upon the needs of theclient. For example, if the client needs to fill out a certain tax formconcerning an IRA rollover, ambient cooperative intelligence process 10may be configured to obtain necessary information based on aconversation between the financial advisor and the client and/orproactively obtain the required information from a datasource accessibleby ambient cooperative intelligence process 10, populate the appropriateform needed to effectuate e.g., the IRA rollover with the data obtainedfrom the datasource, and render (e.g. print) the populated form so thatthe client may execute the same.

Ambient cooperative intelligence process 10 may further be configured toeffectuate the functionality of a digital assistant, wherein ambientcooperative intelligence process 10 may monitor the conversation between(in this example) the financial advisor and the client so that itemsthat were mentioned may be flagged for follow-up. For example, assumethat during the above-described conversation between the financialadvisor and the client that the client stated that they are interestedin setting up 529 college savings accounts for their children and theyasked the financial advisor to provide them information concerning thesame. Accordingly, ambient cooperative intelligence process may enter(e.g. into a client-specific to do list) “Send 529 information to theSmith family”. Additionally, in the event that the client says theywould like to have a follow-up meeting in three weeks to chat about529's, ambient cooperative intelligence process 10 may schedule ameeting within the calendar of the financial advisor for such adiscussion.

Legal Information:

For example, ambient cooperative intelligence process 10 generally(and/or ACD system 54 specifically) may be configured to automate thecollection and processing of legal data that is generated during anencounter in which legal information is discussed. An example of such anencounter may include but is not limited to a meeting between a legalprofessional and a person whom they are representing. For example,ambient cooperative intelligence process 10 may be configured tosupplement/complement a legal professional's knowledge by recommendingstrategies, answering questions and providing advice based on theconversation that the legal professional is having with their client inessentially real time, as well as completing hearing/depositiontranscripts, warrants, court orders/judgements, various applications forthe foregoing and other items, etc.

Benefits: The benefits achievable by ambient cooperative intelligenceprocess 10 when configured to process legal information may beconsiderable. For example and as is understandable, legal professionalsmay not know all things concerning e.g., various legal situations,events and procedures. Accordingly, ambient cooperative intelligenceprocess 10 (when configured to process legal information) may monitor aconversation between the legal professional and the client. Ambientcooperative intelligence process 10 may then utilize natural languageprocessing and artificial intelligence to identify issues/questionswithin the conversation and leverage collective knowledge to providepertinent information to the legal professional.

For example, assume that a deposition is occurring where a defendant ina lawsuit (who is being represented by a first group of attorneys) isbeing asked questions by the plaintiff in the law suit (who is beingrepresented by a second group of attorneys). Accordingly and through theuse of the various systems described above (e.g., audio input device 30,display device 32, machine vision input device 34, and audio renderingdevice 116), ambient cooperative intelligence process 10 (whenconfigured to process legal information) may monitor the conversationbetween the defendant/first group of attorneys and the plaintiff/secondgroup of attorneys. In such a situation, ambient cooperativeintelligence process 10 (when configured to process legal information)may be configured to effectuate the functionality of a courttranscriptionist.

For example, the participants in the deposition may be asked to identifythemselves (e.g. provide name and title). Ambient cooperativeintelligence process 10 may use this information to populate anattendance log concerning the deposition and may be configured to definea voiceprint and/or face print for each attendee of the deposition.

Accordingly and once the deposition actually starts, ambient cooperativeintelligence process 10 may monitor the deposition and may (via theabove described voice prints/face prints) diarize the same, essentiallyreplicating the functionality of a court transcriptionist. Basically,ambient cooperative intelligence process 10 may generate a diary of thedeposition proceeding that reads like a movie script, wherein e.g. eachspoken statement is transcribed and the speaker of that spoken statementis identified (via the voiceprint/face print).

Additionally and through the use of the above-describe natural languageprocessing and artificial intelligence, traditional legal tasks may beefficiently effectuated. For example, suppose that (during thedeposition) an objection is made and a piece of case law is cited as thebasis for the objection. If the non-objecting attorney believes thatthis piece of case law is no longer valid (e.g. due to it beingoverturned by a higher court), the non-objecting attorney may askambient cooperative intelligence process 10 (when configured to processlegal information) to determine the status of the relied-upon piece ofcase law (i.e., whether the piece of case law is still valid or has beenoverturned). Ambient cooperative intelligence process may then providean answer to the non-objecting attorney (e.g., the case is still validor the case was overturned by the 1^(st) Circuit Court of Appeals in2016, which was affirmed by the US Supreme Court in 2017).

Telecom Information:

For example, ambient cooperative intelligence process 10 generally(and/or ACD system 54 specifically) may be configured to automate thecollection and processing of telecom data that is generated during anencounter between a caller and a sales/service representative. Anexample of such an encounter may include but is not limited to atelephone call and/or chat session between a sales/servicerepresentative and a customer who is having a problem with their cabletelevision service. For example, ambient cooperative intelligenceprocess 10 may be configured to supplement/complement a servicerepresentative's knowledge by recommending plans/products,trouble-shooting procedures, answering questions and providing advicebased on the conversation that the service representative is having withtheir customer in essentially real time.

Benefits: The benefits achievable by ambient cooperative intelligenceprocess 10 when configured to process telecom information may beconsiderable. For example and as is understandable, sales/servicerepresentatives may not know all things concerning e.g., various serviceplans, available products, trouble-shooting procedures, and warrantycoverage. Accordingly, ambient cooperative intelligence process 10 (whenconfigured to process telecom information) may monitor a conversation(e.g., voice or text) between the service representative and the caller.Ambient cooperative intelligence process 10 may then utilize naturallanguage processing and artificial intelligence to identifyissues/questions within the conversation and leverage collectiveknowledge to provide pertinent information to the telecom salesperson.

For example, assume that a user of a cable television service is havinga difficult time tuning to one of their pay channels within their cableTV channel list. Accordingly, this user may call up (or message) theircable television service and chat with a customer servicerepresentative. Ambient cooperative intelligence process 10 (whenconfigured to process telecom information) may e.g. utilize caller ID,IP addresses and/or voice prints to identify the caller and obtaininformation concerning their account, their location, their equipment,their service plan, etc.

Assume for this example that the caller explains to the servicerepresentative that they cannot tune their cable box to the desiredchannel. Ambient cooperative intelligence process 10 may e.g. firstconfirm that their current service plan includes the channel that thecaller is trying to access. In the event that the service plan does notinclude such channel, ambient cooperative intelligence process 10 maynotify the service representative (e.g. via a text-based message visibleon a display accessible by the service representative or via an earbud)that the channel is not included in their service plan. Ambientcooperative intelligence process 10 may then provide information to theservice representative concerning which service plans include thechannel about which the caller is inquiring to see if e.g., they want toupgrade/change their plan to one that includes the channel in question.

In the event that the channel is indeed included in the current serviceplan of the caller, ambient cooperative intelligence process 10 maybegin to provide prompts to the service representative concerning atroubleshooting procedure that may be utilized to identify the problem.For example, ambient cooperative intelligence process 10 (via e.g. adisplay or an earbud) may provide the service representative with asequence of steps that the caller can perform in order to (hopefully)rectify the situation. For example, the service representative mayinstruct the caller to first unplug the cable box from the electricaloutlet and let it sit for 30 seconds and then plug it in so that it mayreboot. In the event that this procedure does not fix the problem, thelist provided by ambient cooperative intelligence process 10 mayinstruct the service representative to send a reset signal to the cablebox in question. In the event that this procedure does not fix theproblem, ambient cooperative intelligence process 10 may determine thata new cable box is needed and may assist the service representative inscheduling a service call so that the faulty cable box may be replacedby a service technician.

Retail Information:

For example, ambient cooperative intelligence process 10 generally(and/or ACD system 54 specifically) may be configured to automate thecollection and processing of retail data that is generated during anencounter in which retail information is discussed. An example of suchan encounter may include but is not limited to a meeting between asalesclerk at a department store and a person interested in purchasing aparticular product. For example, ambient cooperative intelligenceprocess 10 may be configured to supplement/complement a salesclerk'sknowledge by recommending products, answering questions and providingadvice based upon the conversation that the salesclerk is having withtheir customer in essentially real time, as well as enabling checkout,completing work order forms, financial/sales agreements, product orderforms, warranty forms, etc.

Benefits: The benefits achievable by ambient cooperative intelligenceprocess 10 when configured to process retail information may beconsiderable. For example and as is understandable, salesclerks may notknow all things concerning e.g., the assortment of products offered andthe location of the same. Accordingly, ambient cooperative intelligenceprocess 10 (when configured to process retail information) may monitor aconversation between the salesclerk and the customer. Ambientcooperative intelligence process 10 may then utilize natural languageprocessing and artificial intelligence to identify issues/questionswithin the conversation and leverage collective knowledge to providepertinent information to the salesclerk.

For example, assume that a customer goes to a local department store andthey are looking for several items, including an electric drill. So thiscustomer approaches a salesclerk and asks them if they sell electricdrills and, if so, where they are. Ambient cooperative intelligenceprocess 10 (when configured to process retail information) may monitorthis conversation and identify the issues that need to be addressedthrough the use of the above-described natural language processing andartificial intelligence. For example, ambient cooperative intelligenceprocess 10 may identify the phrase “electric drill” within the statementmade by the customer and may examine inventory records for thedepartment store and determine that the department store does indeedsell electric drills. Further, ambient cooperative intelligence process10 may determine that the customer is asking about the location of theseelectric drills and, upon checking product stocking charts for thedepartment store, may determine that electric drills are in the hardwaresection (aisle 23, bays 16-20).

Additionally, ambient cooperative intelligence process 10 may beconfigured to address additional questions that the customer may have,such as ‘What electric drills the have that cost under $30?”, “Whatelectric drill has the longest warranty?”, “What electric drills do youhave from DeWalt?” and “Do you have drill bits for drilling intocement?”. When providing answers concerning these questions raised bythe customer, ambient cooperative intelligence process 10 may overtlyprovide the information onto a display screen (e.g. a handheldelectronic device) so that the customer may review the same.Alternatively, ambient cooperative intelligence process 10 may covertlyprovide the information in an earbud so that the salesclerk may verballyprovide the information to the customer.

Further, assume that a family goes into a local wireless carrier storeto inquire about cell phones and cell phone plans. Accordingly andthrough the use of the various systems described above (e.g., audioinput device 30, display device 32, machine vision input device 34, andaudio rendering device 116), ambient cooperative intelligence process 10(when configured to process retail information) may monitor theconversation between the family and salesclerk and provide guidance andinsight with respect to such conversation through the use of theabove-described natural language processing and artificial intelligence.For example, assume that the family asks the salesclerk if there are anysales/promotions on the latest iPhones. If so, ambient cooperativeintelligence process 10 (when configured to process retail information)may covertly provide a list of sales/promotions to the salesclerk viae.g., an earbud assembly or may overly provide a list ofsales/promotions to the salesclerk via e.g., a client electronic device(e.g., a smart phone, a tablet, a laptop, or a display).

Additionally, assume that the family inquires as to what is the bestphone to buy and/or what is the best data plan to be on when you doextensive international traveling. Accordingly, ambient cooperativeintelligence process 10 (when configured to process retail information)may e.g. render a list of applicable phones/data plans on a clientelectronic device (e.g. a smart phone, a tablet, a laptop, or display)so that such options may be reviewed with the salesclerk. Further, inthe event that ambient cooperative intelligence process 10 determinesthat one or more members of the family is interested in a cellulartelephone that is not compatible with the cellular networks in variouscountries around the world, ambient cooperative intelligence process 10may prompt the salesclerk to inquire as to whether this family membertravels to e.g., Countries A, B or C.

Additionally, as ambient cooperative intelligence process 10 may bemonitoring the conversation between the family and the salesclerk,ambient cooperative intelligence process 10 may determine the quantityof cellular telephones they are interested in purchasing. Ambientcooperative intelligence process 10 may then review the variouspromotional plans being offered by the cell phone manufacturers, as wellas any the available data plan options, so that ambient cooperativeintelligence process 10 may present the phones and data plans that aremost advantageous to the family.

Additionally, ambient cooperative intelligence process 10 may monitorthe conversation between the family and the salesclerk to identifyand/or correct any mistakes or misrepresentations that the salesclerkmay have inadvertently made. For example, if the user said that theyoften travel to Country X and they are in the process of purchasingCellular Telephone Y (which is not usable within Country X), ambientcooperative intelligence process 10 may covertly notify (e.g. via anearbud) the salesclerk that Cellular Telephone Y will not functionproperly within Country X.

General:

As will be appreciated by one skilled in the art, the present disclosuremay be embodied as a method, a system, or a computer program product.Accordingly, the present disclosure may take the form of an entirelyhardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,the present disclosure may take the form of a computer program producton a computer-usable storage medium having computer-usable program codeembodied in the medium.

Any suitable computer usable or computer readable medium may beutilized. The computer-usable or computer-readable medium may be, forexample but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. More specific examples (a non-exhaustive list) ofthe computer-readable medium may include the following: an electricalconnection having one or more wires, a portable computer diskette, ahard disk, a random access memory (RAM), a read-only memory (ROM), anerasable programmable read-only memory (EPROM or Flash memory), anoptical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a transmission media such as those supportingthe Internet or an intranet, or a magnetic storage device. Thecomputer-usable or computer-readable medium may also be paper or anothersuitable medium upon which the program is printed, as the program can beelectronically captured, via, for instance, optical scanning of thepaper or other medium, then compiled, interpreted, or otherwiseprocessed in a suitable manner, if necessary, and then stored in acomputer memory. In the context of this document, a computer-usable orcomputer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited tothe Internet, wireline, optical fiber cable, RF, etc.

Computer program code for carrying out operations of the presentdisclosure may be written in an object oriented programming languagesuch as Java, Smalltalk, C++ or the like. However, the computer programcode for carrying out operations of the present disclosure may also bewritten in conventional procedural programming languages, such as the“C” programming language or similar programming languages. The programcode may execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through a local area network/a widearea network/the Internet (e.g., network 14).

The present disclosure is described with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the disclosure. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, may be implemented by computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer/special purposecomputer/other programmable data processing apparatus, such that theinstructions, which execute via the processor of the computer or otherprogrammable data processing apparatus, create means for implementingthe functions/acts specified in the flowchart and/or block diagram blockor blocks.

These computer program instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures may illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustrations,and combinations of blocks in the block diagrams and/or flowchartillustrations, may be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present disclosure has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the disclosure in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the disclosure. Theembodiment was chosen and described in order to best explain theprinciples of the disclosure and the practical application, and toenable others of ordinary skill in the art to understand the disclosurefor various embodiments with various modifications as are suited to theparticular use contemplated.

A number of implementations have been described. Having thus describedthe disclosure of the present application in detail and by reference toembodiments thereof, it will be apparent that modifications andvariations are possible without departing from the scope of thedisclosure defined in the appended claims.

What is claimed is:
 1. A computer-implemented method executed on acomputing device comprising: generating a three-dimensional model of atleast a portion of a three-dimensional space incorporating an ambientcooperative intelligence (ACI) system via a video recording subsystem ofan ACI calibration platform; generating one or more audio calibrationsignals for receipt by an audio recording system included within the ACIsystem via an audio generation subsystem of the ACI calibrationplatform, wherein the one or more audio calibration signals includes thethree-dimensional model of at least the portion of the three-dimensionalspace; and autonomously positioning the ACI calibration platform withinthe three-dimensional space via a mobile base assembly of the ACIcalibration platform based upon, at least in part, the three-dimensionalspace of the three-dimensional model.
 2. The computer-implemented methodof claim 1 wherein the video recording system of the ACI calibrationplatform is configured to interface with an object datasource thatdefines at least one of one or more stationary objects located withinthe three-dimensional space, wherein the three-dimensional model isfurther configured to define one or more interaction zones within thethree-dimensional space.
 3. The computer-implemented method of claim 1further comprising: autonomously cleaning at least a portion of thethree-dimensional space via a cleaning assembly of the ACI calibrationplatform.
 4. The computer-implemented method of claim 1 wherein the ACIcalibration platform is configured to be manually positioned within thethree-dimensional space.
 5. The computer-implemented method of claim 2wherein the one or more interaction zones includes a patient examinationzone proximate the at least one of the one or more stationary objectswithin the three-dimensional space.
 6. The computer-implemented methodof claim 1 wherein the three-dimensional model is further configured todefine at least one of: one or more subspaces within thethree-dimensional space; one or more objects within thethree-dimensional space; one or more features within thethree-dimensional space; and one or more noise sources within thethree-dimensional space.
 7. The computer-implemented method of claim 1wherein the one or more audio calibration signals includes one or moreof: a noise signal; a sinusoid signal; and a multi-frequency signal. 8.A computer program product residing on a non-transitory computerreadable medium having a plurality of instructions stored thereon which,when executed by a processor, cause the processor to perform operationscomprising: generating a three-dimensional model of at least a portionof a three-dimensional space incorporating an ambient cooperativeintelligence (ACI) system via a video recording subsystem of an ACIcalibration platform; generating one or more audio calibration signalsfor receipt by an audio recording system included within the ACI systemvia an audio generation subsystem of the ACI calibration platform,wherein the one or more audio calibration signals includes thethree-dimensional model of at least the portion of the three-dimensionalspace; and autonomously positioning the ACI calibration platform withinthe three-dimensional space via a mobile base assembly of the ACIcalibration platform based upon, at least in part, the three-dimensionalspace of the three-dimensional model.
 9. The computer program product ofclaim 8 wherein the video recording system of the ACI calibrationplatform is configured to interface with an object datasource thatdefines at least one of one or more stationary objects located withinthe three-dimensional space, wherein the three-dimensional model isfurther configured to define one or more interaction zones within thethree-dimensional space.
 10. The computer program product of claim 8further comprising: autonomously cleaning at least a portion of thethree-dimensional space via a cleaning assembly of the ACI calibrationplatform.
 11. The computer program product of claim 8 wherein the ACIcalibration platform is configured to be manually positioned within thethree-dimensional space.
 12. The computer program product of claim 9wherein the one or more interaction zones includes a patient examinationzone proximate the at least one of the one or more stationary objectswithin the three-dimensional space.
 13. The computer program product ofclaim 8 wherein the three-dimensional model is further configured todefine at least one of: one or more subspaces within thethree-dimensional space; one or more objects within thethree-dimensional space; one or more features within thethree-dimensional space; and one or more noise sources within thethree-dimensional space.
 14. The computer program product of claim 8wherein the one or more audio calibration signals includes one or moreof: a noise signal; a sinusoid signal; and a multi-frequency signal. 15.An ambient cooperative intelligence (ACI) calibration platformcomprising: a video recording subsystem configured to generate athree-dimensional model of at least a portion of a three-dimensionalspace incorporating an ACI system; an audio generation subsystemconfigured to generate one or more audio calibration signals for receiptby an audio recording system included within the ACI system, wherein theone or more audio calibration signals includes the three-dimensionalmodel of at least the portion of the three-dimensional space; and amobile base assembly configured to autonomously position the ACIcalibration platform within the three-dimensional space based upon, atleast in part, the three-dimensional space of the three-dimensionalmodel.
 16. The ACI calibration platform of claim 15 wherein the videorecording system of the ACI calibration platform is further configuredto interface with an object datasource that defines at least one of oneor more stationary objects located within the three-dimensional space,wherein the three-dimensional model is further configured to define oneor more interaction zones within the three-dimensional space.
 17. TheACI calibration platform of claim 15 further comprising: a cleaningassembly configured to autonomously clean at least a portion of thethree-dimensional space.
 18. The ACI calibration platform of claim 15wherein the ACI calibration platform is configured to be manuallypositioned within the three-dimensional space.
 19. The ACI calibrationplatform of claim 16 wherein the one or more interaction zones includesa patient examination zone proximate the at least one of the one or morestationary objects within the three-dimensional space.
 20. The ACIcalibration platform of claim 15 wherein the three-dimensional model isfurther configured to define at least one of: one or more subspaceswithin the three-dimensional space; one or more objects within thethree-dimensional space; one or more features within thethree-dimensional space; and one or more noise sources within thethree-dimensional space.
 21. The ACI calibration platform of claim 15wherein the one or more audio calibration signals includes one or moreof: a noise signal; a sinusoid signal; and a multi-frequency signal.